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ABSTRACT 

A study for computer selection at Andrews University, Berrien 
Springs, Michigan, was completed in February 1973. Methods used 
in the study are described and critiqued. Recommendations are made 
based on the experience gained in the study. The computer is to 
serve as a combined academic and administrative machine for a 
university with total enrollment of 2100 students. Total system 
cost is nominally $10,000 per month. 
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Section 1 



INTRODUCTION 



1,1 SCOPE OF STUDY 

This report describes a computer selection study conducted at Andrews 
University, Berrien Springs, Michigan, between August 1972 and Feb- 
ruary 1973. The study was exhaustive and involved about one man-year 
of effort. In addition to .describing the study, this report incor- 
porates the lessons learned from the study. It is hoped that others ma 
may benefit from this approach. 

Although all careful selectison studies have many common elements, 
. there is no single method adequate for all situations. Factors such 
as institutional size, required user applications, user sophistica- 
tion, and financial limitations affect the methods used for a study. 
Some factors affecting the Andrews University study are presented here 
to set the tone for the description of the study • Andrews Univarsity is 
a private institution with a total enrollment of 2100 students (college, 
graduate school, and seminary). Size alone dictated a combined oacademic/ 
administrative computer center. (An investigation was later made to ver-^i 
ify this tentative decision). In order to provide for work-study programs, 
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several commercial enterprises closely associate with the University., 
Thus the range of computer services required spans most commercial ap- 
plications. The IBM 360/22 has handled most of these applications vjell. 
Unfortunately, the IBM, 360/22 has not been adequate for most academic 
applications (necessitating considerable expenditures for outside ser- 
vices during the past two years). One major purpose for the selection 
study was to find a system which could handle the existing 500 COBOL 
programs used in administration while satisfying the bulk of academic 
requirements. These requirements biased the study in favor of versa- 
tility rather than raw throughput and helped, establish requirements for 
main memory user area. The acceptance of a computing system by students 
and faculty is largely dependent on the ease of use of the system. The 
selection committee found the most significant single factor in ease 
of use to be adequate provision for timesharing. Administrative users 
previously had almost exclusive use of the IBM 360/22. To continue to 
satisfy those users acceptable concurrent batch and timesharing proces- 
sing capability is required. 

> .. • - ■ . ' 

Andrews University offers occupational education courses in keypunch 
and verifier operation, computer operation^ programming and introductory 
systems analypjis. A B.S. in information science will be offered begin- 
ning with the Fall 1973-1974 term. Concentrations in information science 
are also offered as parts of other curriculums. Such offerings obviously 
require adequate hardware support. 
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1.2 INITIAL CONSIDERATIONS / 

There are several preliminary questions which must be met b<eif ore making 
a coramitment to put forth the effort required to-conduct a careful selec- 
tion study: 

• Does a necessity exist for a computer or an upgrade? 

9 Is there willingness to make a thorough objective study 
of feasibility, hardware selection, and acquisition? 

• Is there bias on the part of computer management, top 
management, or members of the board to procure nothing 
but brand "X" equipment? Such entrenched brand, loyalty 
can dest?:oy any benefit which could be gained from an 
otherwise objective study. 

1.3 THE COMMITTEE 

It is important to have representatives of the systems analysis/programming 
and operations staffs on the selection committee to ensure adequate tech- 
nical expertise, and acceptance of the findings of the study by the comput- 
ing center staff. In a university environment faculty members with sophi- 
sticated knowledge of computing systems' represent a rich resource. Admini- 
stration representation is also important. 

Early financial, guidance is very helpful in setting cost criteria* In 
the final stages of the study (acquisition analysis) and contract nego- 
tiations, top management will feel more secure if they are adequately 
represented. Administration should be in close contact with the study 
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throughout Its progress . 



*It should be expected when It appears that a sale may be lost, certain 
vendors will try to high pressure top administrators with sales and 
scare tactics. Administrators should be aware of such tactics. At 
Andrews one vendor attempted to convince top administration that the 
study was not adequate. Being forewarned, they rejected the ploy. 
Since top management had been kept well-informed of the progress of the 
study, and felt a sense of participation, there was considerable confi- 
dence in the study. Since most administrators are laymen in the comput- 
ing field and have neither sufficient time or interest to become experts, 
it is important that they have confidence In the technical judgement of 

those actually conducting the study, 

\ • ■ 

1.4 METHODS USED IN THE SELECTION STUDY 

The following methods of study and sources of information were- used 
in the Andrews selection study (or are suggested for use) : 

• Faculty and administration contacts. 

• Vendor contacts. 

• Written Request for Proposal. 

• Weighted point rating system. 

0 Datapro 70 and Auerbach Computer Technology Reports . 

• Trade journals and publications. 

• Reference accounts and other users*. 
. -!.J. i . ♦ Consultants. < 
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• Benchmark tests. / 
J ; • Contract study. 

• Corporate stability analysis. 

j • Detailed procurement analysis. - 

/ j • Demonstration trip. 
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^Section 2 
FEASIBILITY STUDY- -~ 

i ■ -I 

2.1 INTRODUCTION 

■ • 

The purpose of this phase of a selection study Is to establlsh-requlred 
applications , determine acceptable mlnimums for capabilities and support 
services, and set system criteria. In a coinmerclal environment j a feasl- 
billty study must demonstrate financial returns in excess of cost (unless 
other factor^s such as prestige, etc., are highly rated). In the univer- 
sity environment a feasibility study should ensure that the initial pro- 
curement meets, but does not greatly exceed,, institutional nefeds. Even 
in the university environment it is often possible to demonstrate £ln- 
ancial returns in excess of cost. (See Appendix C). 



It is imperative that this portion of the selection process be completed 

prior to making vendor contacts. Vendors frequently offer to "do the 

'^feasibility study;" however, the user risks a serious bias. The obvious 
% , , ' ' ■ ' ■ ' . . ■ ; . 

bias would be over-procurement of hardware. Less obvious p but as Impor- 

tant, assessment of requirements may be biased to fit the strengths and 

weaknesses of the specific vendor's products. 



I 
I. 



2.2 REQUIRED APPLICATIONS 
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It was the consensus of the selection committee that the proper method 
for defining hardware requirements was to first define the required 
applications, then determine software necessary to implement those ap- 
plications, and only then establish hardware requirements necessary to 
Implement the software. Meetings were held with the Individual faculties 
of most academic departments to obtain directly Information on required 
and desired applications and to project requirements for five years* 
(A significant byproduct of this interaction, was the stimulation of many 
faculty members.) The raw data collected In these meetings Is attached 
as Appendix A. The selection committee evaluation of the data Is attached 
as Appendix B. 

Similar contacts were made with administrative users and potential users. 
It rapidly became cleat that the academic usage placed the greatest de- 
mands on potential system facilities. 

2.3 ECONOMIC ANALYSIS V 

At this point financial guidelines were established. A preliminary 
study of methods of financing was made to find the lowest cost of money. 
A reasonable arid uniform estimate of procurement costs for proposed 
systems was based on this preliminary study for purposes of the hardware 
selectlori Study. , ,| 

Relevant cost factors were identi-fied so that proper questions could 
be asked of ^ vendors., Certain factors (eg., power, air conditioning. 



bundling policy, etc.) were Included with cost factors for this pur-- 
pose. (See Section 4), * ^ 

2.4 CRITERIA AND DESIRABLE FEATURES 

In establishing minimum requirements It Is essential to maintain a 
careful balance. Criteria should be made knovm to vendors on the Ini- 
tial contact. This Is only fair. Vendors may fall to respond, or 
respond with little enthusiasm, if criteria are Improperly stated. 
Of course, if the vendor does not have a product which meets Institu- 
tional needs it Is just as well that no proposal be made. However, 
"overly restrictive criteria may conceivably eliminate a viable system. 
Similarly, overly loose criteria will not be of help In the sifting 
process. Criteria used in the Andrews University study are Included 
as Appendix D. • , .r 

I - ■ ■ ■ : 

The statement of criteria should Include the maximum ailowable total 
monthly cash flow for system procurement, maintenance charges, and 
software fees. 

One of the major oversights made In the Andrews study was failure to 
clearly specify, as a criterion, that only systems and software 
installed prior to July 1973 would be considered. The date should have 
been set to ensure that systems could be operational and somewhat de- 
bugged before being proposed. As It was, considerable effort was 



expended In studying a "paper tiger" that will probably not be 
available for installation until well after the specified installation 
deadline. Evc ..f slippage in that system had' not occurred, it is 
not desirable to be a pioneer. Present judgement is that^ optimum 
system age (including operating system and essential processors) 
is between one and four years after first installation. 
.1 

A clear statement of features which would be desirable but not essen- 
tial should be submitted to vendors at the first contact. 

The above information is necessary to obtain a valid response from a 
vendor. Unless adequate guidance is given in advance proposals tend 
to be very sales oriented and unresponsive to institutional needs. 

2.5 REQUEST FOR PROPOSAL 

A written request for proposal (RFP) should be prepared prior to 
contacting vendors. The RFP should contain at least the following 

A statement of the types of functions expected of the 
system. 

A statement of required vendor support functions. 
A proposal format. 

J 

Summary forms to be filled, in by vendor. Forms should 
be detailed enough to provide sufficient information for 
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elements: 



hardware selection (See Section 3) . Failure to provide 
such forms (or, as In the case of Andrews, failure to 
provide forms of sufficient detail) may result In being 
deluged with sales oriented papers. Let the vendor 
extract the pertinent data from his sales literature! 
Some caution must be taken to keep the forms unambig- 
uous* Too much unnecessary detail may discourage a 
vendor from submitting a proposal. 

• Policy matters. Evaluation techniques and standards 
(Including criteria and desirable features) should be 
available to vendors. They may offer valid crltlGism 

of the evaluation technique if It is revealed. A state- 
ment should be included to the effect that evaluation 
techniques may be modified at the discretion of the study 
committee. Benchmark, proposal presentation, and demon- 
stration policies and dates should be stated. 

• Terms and Conditions. Non-standard terms and conditions 
required or desired by the institution should be stated. 

• Expected time frame for contract award and debriefing 
should also be stated. 

Although Andrews University did develop an RFP, it was not as complete 
as that described above. The final results of the study were probably 
not affected; however, workload for vendors and Andrews was clearly 
Increased. 



i V- 

2.6 VENDOR CONTACT ^ 

I 

At this point in the process all prospective vendors should be contacted. 
The RFP should be presented to all prospective vendors in a joint brief- 
. ing session. 
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Section 3 
HARDWARE SELECTION STUDY 

3.1 INTRODUCTION 



In addition to hardware capabilities this portion of the study 
considers system growth potential, operating system characteristics, 
language processors, conversion support, training support, maintenance 
support, and software support. 

There were 22 initial system proposals. In addition, the prelimi- 
nary economic study had located seven outside sources of funding. 
After a brief review, eight system proposals were chosen for detailed 
evaluation. Although Control Data Corporation elected not to sub- 
mit a proposal, an excellent cross section of vendors was represented 
in the eight systems: Burroughs, Digital Equipment Corporation, 
Hewlett^-Packard, Honeywell Information Systems, International Bus- 
iness Machines, National Cash Register, Univac Division of Sperry 
Rand , and Xerox. 

in order to allow valid comparisons between proposed systems, each 
vendor was asked to supply information which would allow limited 
reconfiguration by the Selection Committee. Before beginning detailed 
comparative evaluations between the eight systems remaining, an effort 
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was made to reconfigure those systems to be nearly equivalent In gross 
capability • 

J' 

Every attempt was made to keep the system cost from biasing the capa- 
bility portion of the study. The best pricing formula from the eight 
available outside sources of funding was chosen as a common basis of 
cost comparisons* One vendor did have a more advantageous financing 
policy available which was applied to that equipment. It was expected 
that various vendors would reprice their proposals at later stages of 
the study ("What do I have to do to get the business'^ syrtdrome). 
For this reason undue emphasis was not placed on systems cost at this 
point in the study. Certain vendors use fixed price schedules, others 
rebld over and over« In the Andrews study one vendor resubmitted 
proposals three times to Include enhancements , each time dropping the 
price. He dropped the price on two additional occasions. The initial 
RFP should contain a firm date beyond which reblds will not be accepted. 
In spite of the necessity for avoiding overall bias due to cost con- 
siderations, it is crucial to consider cost from the outset. Even with 
a clearly stated maximum cost criterion, several vendors submitted one 
or more proposals for higher cost systems. 

Another major caution is avoid the "low ball" sales technique used ty 
one vendor in the Andrews study. He proposed a system which was in- 
adequately configured to perform required tasks in order to meet the 
price criterion e If the user should accept such a proposal, it is 



usually only a short time until necessary additions have brought 
the system cost well outside the guidelines set by top management. 



3.2 PROPOSAL RATING METHOD 
3.2,1 Summary 

Information provided by vendors was categorized and evaluated in de- 
tail by the coimnittee. Appendix E contains detailed listings, by 
major categories, of the information evaluated. Items of information 
were arranged in columns adjacent to a weighting factor column and a 
column for each proposal under evaluation. After the raw data had 
been summarized in a similar format , the items of information were 
evaluated row by row. Each item* was evaluated for each proposal on 
a scale of zero (unsatisfactory) to seven (outstanding). That score 
was multiplied by a weighting factor to arrive at point scores for each 
item analyzed. To avoid bias, th6 weighting factors were agreed upon 
prior to rating proposals. The point scores were grouped into five 
major categories — these were then reweighted. Thelt: contributions to 
a total point score was as follows: 

• Hardware Capabilities (20%), 

• Growth Potential (15%). 

• Operating System (20%). 

• Language Processors (20%). 

• Conversion, Maintenance and Software Support (25%). 
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IfSximum possible score for the Andrews study was 700, The highest 
score assigned was 500. 



3. 2, 2 Comments Regarding the Rating System 

The major strength of the rating system Is that It permits the many 
characteristics of complex systems to be broken Into small units 
which can be realistically evaluated. The use of such a rating system 
allows easy preparation of questionnaires for submission to vendors 
as part of an RFP (let the vendors search through the boilerplate) . 
Although It may at first appear that" reliance on unverified sales 
presentations Is a weakness of the method, a major advantage does exist: 
it Is very unlikely that the "best choice" system will be overlooked. 
(VJhat salesman will seriously underestimate the capabilities of his 
product line?) It was left to the benchmark programs, reference 
accounts, demonstrations, and acceptance test standards to ensure 
that a system was chosen on the basis of demonstrable characteristics 
rather than a well made sales presentation. 

The response of each vendor to the RFP and benchmark should be evaluated 
as ah indication of support to be expected after installation. The 
rationale for such a rating is the eagerness with which vendors respond 
should only be expected to decline after a contract is signed ♦ Inadequat 
response to an RFP or benchmark request should be interpreted as a danger 
signal. 



Overall acceptability of the study is improved when objective data 
is evaluated. However, it is virtually impossible to do a complete 
analysis without including subjective data- At least by using a 
weighted point method of analysis the subjective evaluations can be 
made in small carefully discussed units. 

It is staggering to attempt an adequate evaluation of applications 
software, including that available from each user group, for a large 
group of proposals. Andrews, as part of the initial rating, found that 
most vendors in one way or another provided most required packages. ^ 
A detailed analysis of the quality of such packages was made in a 
later phase of the study. 

.Throughput rating schemes (eg., Gibson Mix and SCERT) are useful in 
the initial rating period as long as they are not relied upon too heavily. 
The Andrews study used a throughput model only to evaluate the sensiti- 
vity of system throughput to various hardware options — no-great emphasis 
was placed on results in the actual evaluation ^ 

Perhaps the greatest problem is rating proposals is comparing systems 
which are inherently different in certain characteristics (the "apples 
to oranges" problem). Great care and exercise of judgement must be 
used in this area* Largely for this reason, a series of formal pre- 
sentations was scheduled to. permit one last opportunity for all 
selection committee members to have direct interaction with all 
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vendors whose proposals were being rated. Certain missing data were 
obtained; and, of greater Importance, clarification of the precise 
meaning of certain data was obtained. Such meetings should be sche- 
duled with Individual vendors before attempting to finalize point 
rating system conclusions. 

Although use of the weighted point rating method doe^ result in a 
rather concrete quantitative result, lending an aura of precision 
ro tnc whole study, it is worthwhile to evaluate expected limits of 
5^rr>r present in the results. 

In conducting any selectl/on study the matter of personal bias can be 
a serious problem. By using a committee to review detailed data point 
by. point the personal bias problem is reduced. Even when each com- 
mittee member attempts to be carefully objective, the problem of rating 
a specific area high (or low) because the system in general shows a 
pattern, of high (or low) ratings must be ^acognlzed (the "halo" effect). 
Apparently the best method of avoiding this problem is to ensure that 
all committee members are consciously and continually aware of its 
existence. 

Considerable committee effort was spent in identifying items of data 
to be. evaluated and setting weighting factors. It was difficult to 
achieve consensus in choice of weighting, factors; however, agreement 
was reached. After evaluating all data, a sensitlsrity study was made 
which Included all of the major variants in weighting factors which 



had been proposed. It was discovered that the rating method was 
insensitive to changes In the variant weighting factor patterns proposed 
at Andrews. ^ 

In spite of the difficulty of gathering a set of data which accu- 
rately and adequately pertained to the specific environment, Andrews 
found the weighted point rating method to be highly satisfactory. 
Sales-pitch had been largely omitted from the rated data. After an 
intensive study of the top rated proposals very little disagreement 
was found between the Initial and final ratings. Considerable effort 
has been expended in checking the validity and reliability of the rating 
system (Andrews hopes to share experience gained in the present study 
with approximately 80 other denominational computing centers, about 
20 of which are presently engaged in or contemplating selection studies) 
The evaluation of the rating system in general, is that it is not the 
complete answer to a selection study, but does represent a highly sat- 
isfactory method for initial screening of proposals. 

. . . .J 

3.3 PUBLICATIONS 

Datapro 70 > Auerbach Computer Technology Reports , and various other 
trade journals arid publications were used to verify and supplement 
information furnished by vendors. 

Standardized reports are Invaluable in sifting sales material. 



Convenient suniraaries are presented In a standard format which permits 
system to system comparison between vendors. In a few cases , salesmen 
were not responsive to technical questions ; and in those cases standard' 
ized reports were used to obtain data. ^ 

3.4 OTHER USERS 

Liberal use was made of information provided by reference accounts 
furnished by vendors as well as Information obtained from users 
located by other means. Although some favorable bias might be ex- 
pected from reference accounts, the candor and relatively unbiased 
observations of users was extremely valuable in both verifying and 
discounting various vendor claims. 

Users were especially helpful in assessing convenience and acceptance, 
staffing requirements, quality^^of^applications packages, adequacy of 
support (all areas of support), vendor response'to user's' group, 
user's group activities and benefits, and system management features. 
One technical area which Andrews found almost impossible . to evaluate 
except through user's comments was the degradation in batch throughput 
and terminal response time 'to be expected with various numbers of 
concurrent timesharing users* It is difficult to develop a benchmark 
test (which vendors will agree to_.perform! ) which measures such de- 
gradation in a uniform manner for all vendors. 

In requesting reference accounts it is important to ask each salesman 
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to select accounts who are using systems similar in configuration 
to that proposed and who have at least similar system. requirements. 
The first questions asked of other users should also concern these 
matters • 

Andrews made it a policy not to call any account on a reference account 
list without specific clearance from the appropriate sales representa-. 
tive. Users located via other Tnethods were contacted freely. Although 
vendors were informed of comments, derived from users, the exact source 
of - derogatory information was withheld to ^prevent any possibility of 
embarrassment to the source. 

3,5 CONSULTANT 

In seeking out a consultant, whether the seeker is sophisticated or 
not, there is one hazard: "A consultant," according to a recent 
definition, "is somebody who says he is." Proper use of a consultant's 
talents require careful thought. An excellent discussion concerning 
the choice and use of consultants is contained in Datapro 70 * 

Use of a consultant does not necessarily imply great expense* In^the 

academic community one may find others who are willing to share 

y ■ 

experience at no charge. 



The advantages of using a qualified and objective person from outside 
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of the organization conducting the selection study are numerous and 
sometimes obvious (eg., experience, objectivity, lack of "political" 

4 

pressure, etc*,). One advantage that should not be underestimated is 
the effect on top management. Even an exhaustive and objective study 
will benefit from the stamp, of approval of a qualified outside expert 
("A* prophet is not without honor, save in his own country, and in his 
own house.") . 

Even if acceptance by top management is not a problem, the f indings ^ 
and advice of a consultant (or consultants — Andrews used two) can be 
valuable. ^ * 

3.6 BENCHMARK TESTS . ' 

The classical benchmark test is based, on a- known pr- estimated job 

mix and test programs intended to simulate that mix. Frequently, 

the philosophy of the benchmark evaluation leads to "fastest is best." 

The Andrews benchmark was a demonstration as much as a benchmark 

(in the classical sense). Although throughput estimations were derived 

from the benchmark, the principle purpose of the study was to verify 

the versatility of system hardware and software. Special emphasis was 

placed on evaluating. suspected weaknesses in specific systems, 

A perennial problem in benchmark tests is the. tendency for vendors to 
run the tests on a system faster or more fully configured than the '' 
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proposed system, Andrews solved this problem by requiring the 
benchmark tests to be rerun on the installed system as an acceptance 
test, and by tying payment for the system to adequate performance on 
the acceptance tests • (See Section 5 for terras and conditions. The 
Introduction to the benchmark request is included as Appendix F.) 

The Andrews benchmark test was not distributed until after the initial 
proposal ratings had been completed. Since the vendor investment in 
a benchmark may be very large, Andrews felt it would be unfair to 
ask vendors to execute the benchmark without first giving some indica- 
tion of results of the preliminary rating. For that reason two letters 
of transmittal were used.. One was given to vendors whose proposals 
were still of major interest, stating that a benchmark must be per- 
formed for the proposal to continue to receive consideration. (This 

letter was sent to four vendors; including the vendor with the machine 

r ■ 

which looked good on paper, but was clearly not demonstrable.) The 
second letter was given to vendors whose proposals had been elimir 
nated on the basis of performance criteria, stating that their propo- 
sal had been tentatively eliminated, but that Andrews would reconsider 
that evaluation should a benchmark reveal the system proposed did meet 
or exceed all criteria. (This letter was sent to four vendors.) 

Four of the eight benchmarks requested were completed. For the pro- 
posals tentatively eliminated, three vendors responded. (The vendor 
with the "paper tiger" could not respond. Another vendor with a very 
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highly rated machine who had been evaluated as being marginal in 
support capability, responded 32 days late.) For the group of propo- 
sals tentatively eliminated, one vendor responded. (The benchmark 
from that vendor did bear out prior conclusions of the selection comm- 
ittee. One other vendor in this group responded — not with a benchmark, 
but with a letter to the President of the University which attempted 
to discredit the study. Be prepared!) 

It is imperative that communications be excellent in the request for 
benchmark. The tests must be run as specified if results of the bench- 
mark are to permit valid comparisons between vendors. A representative 
of the selection committee should monitor the performance of each test. 
In the Andrews study direct monitoring was not feasible; however, 
an extended briefing session was held with each vendor to explain 
the purposes of all tests, execution methods, term definitions, and 
evaluation methods. Even with such care, some vendors did not perform 
certain parts of the benchmark as desired. 

Each vendor was given the opportunity to use spooling and other similar 
features to optimize his benchmark performance. Each vendor was also 
invited to submit further results which might show desirable machine 
features. 

In drafting a request for benchmark, as in drafting an RFP, it is vital 
to avoid making explicit or implied commitments which may limit later 
freedom of action. 



Section 4 
ACQUISITION STUDY 

4.1 INTRODUCTION 

Each proposal was fairly clearly ranked on the basis of previous study. 
This portion of the study Involved very intensive analysis of the top 
rated systems (Ideally only one or two systems should be evaluated 
In this phase) . 

4.2 APPLICATIONS SOFTWARE 

A detailed evaluation of available applications packages was made for 
the top rated systems. 

4.3 STANDARD CONTRACTS 

Standard rental » purchase, and maintenance contracts as well as 
standard software licensing agreements should be carefully examined. 
Review of standard contracts by an attorney should be done as an aid 
in establishing terms and conditions. The final contracts must be 
reviewed by an attottiey prior to execution. 



4.4 TERMS AND CONDITIONS 



Non-standard tfemns and cbtlditlons desired or required by the Institu- 
tion should be listed. Appendix G lists non-standard terms and condi- 
tions developed by the Andrews ^tudy. 

Certain terms and conditions may be desirable but not strictly nec- 
essary^ It is worthwhile to Include such terms and conditions in the 
list as "trading material" during contract negotiations. 

4.5 STABILITY ANALYSES 

A study of the corporate economic stability of certain vendors was 
conducted by a member of the business and administration department 
faculty who teaches portfolio management. In particular, investi- 
gation was made as to the current status and Importance of computer 
operations with respect to the total corporation. The reason for 
such a study was to help assess the risk of seeing the corporation 
bow out of the computer business during the lifetime of the machine. 

Further study was made of the stability of the proposed product 
lines within the total vendor product lines. 

^•6 FINAL CONFIGURATION REVIEW 

Before beginning contract negotiations, a very careful review of the 
system configuration was made. Special attention was given to possible 
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growth patterns based on that configuration. Growth which is trans- 
parent to program systems is an important goal. It is easily possible 
to have more real value tied up in software and training than in hard- 
ware . • 

4.7 PROCUREMENT ANALYSIS 

After the configuration is fixed various methods of procurement 
may be analyzed. Advantages and disadvantages of certain methods 
of procurement are outlined in Appendix H. 

A. 7.1 Cost Factors 

Tn addition to the obvious rental, lease, or purchase cost and main- 
tenance cost, careful attention must be paid to the following costs: 

• Software costs. 

• Conversion costs (eg., software and file conversion, 
parallel run costs, initial training cost, etc.). 

• Installation costs (eg., site preparation, transporta- 
tion, initial acquisition costs for disk packs, etc.)* 
Site preparation estimates should be made in consulta- 
tion with the vendors preinstallation team. A detailed 
analysis of space, power, and air-conditioning require-^ 
ments should be made prior to contract negotiations. 

. . • Staffing costs. 
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• Future costs (eg., field expansion charges, etc.). 

• Cost removing and disposing of present system, if any. 

• Cost of documentation. 

• Continuing costs of training. 

4.7.2 Other Factors 

This phase of the study must also include careful attention to the 
following factors: 

• Installation date commitment. 

• Average time between system installation and acceptance. 

• Limitations on terminals or specialized peripherals which 
may be attached to the system. 

• Limitations on the use of the system (eg., extra shift 
operation, selling of outside services, etc.). 

• Limitations on maintenance of systems analyst support. 

• Ease of conversion* 

4.8 DEMONSTRATION 

Prior to entering contract negotiations a trip was made to an installa- 
tion similar to that recommended by the selection committee. The trip 
permitted the selection committee to have a first hand look at equipment 
and ask final questions. Representatives of the business office, 
registrar's office, and administration also made the trip to obtain 



evaluations of the system from their counterparts. 



Had more time existed, visits would have been made to all vendors. 
Since all could not be visited, it was decided to reserve a demonstr 
tion visit only for the recommended vendor. 



Section 5 

USER-VENDOR RELATIONS AND CONTRACT NEGOTIATIONS 
5.1 INTRODUCTION 

Several philosophies exist for doing a selection study. Andrews chose 
to run a very open study. Vendors had access to the Director of the 
computing center at any time. Evaluation status was available to any 
vendor at any time. Short of revealing proprietary Information, facts 
were not held back. This type of study Involves more work than a 
study in which vendors are "locked out" except during specified times; 
however, there are several advantages. The vendors may critique the 
evaluation procedures during the evaluation* The vendor is permitted 
to respond to derogatory Information. By allowing rebids and frequent 
vendor contacts the search for the best system at the best cost is 
enhanced, but at the cost of considerable effort. The Andrews study 
was probably too open. 

5-2 COMMUNICATIONS 

Good communications, throughout the study, are essential. The following 
points deserve special mention: 

• Ground rules should be set early and in writing. 
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An attitude of fairness to all vendors is crucial. 
Make no special connnitments to any vendor (even the 
installed vendor). 

Communicate equivalent information to each vendor. 
This is one of the principle reasons for using a written 
RFP and joint briefing session. 

5.3 VENDOR TACTICS 

Certain vendors will attempt to sell directly to top management — 
especially of it appears that the selection committee study is not 
favoAng their proposal. This may be the best argument for a closed 
study. Top management should be briefed to expect such contacts. 

5.3 CONTRACT NEGOTIATIONS 

Although one major vendor will not write non-standard contracts 
except for government agencies, most vendors will at least consider 
the individual needs of potential customers. A careful and complete 
consideration of this problem was made as part of the Andrews study. 
Every available guarantee and protection possible for the user should 
be included in the final contract. 
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Appendix A 
NEEDS AND DESIRES 



General 



All Departments: 

Grading; Name, Address and Bibliographic Files; Analysis of 
Surveys; Def^artmental Records and Inventories; Text Editing; 
Student Profiles (for Counseling); Information Retrieved (eg., 
ERIC) 



, ^ 

Concordance (COM), Textual Analysis, Syntax Analysis j Text Editing 



College and Graduate School 



Agriculture: 

Genetics Study, Nutritional Analysis 

Art: 

Computer Art 

Behavioral Science: 

Statistical Analysis (see Math and Statistics) of Experiments and 
for Testing, Speech Simulation, CAI 

Biology: * 

Genetics Simulation, Classification Studies, CAI 

Business Administration: 

Accounting, PERT, Linear Programming, Inventory Control, Gaming, 
CAI 

Chemistry: 

Experimental and Theoretical Analysis (Kinetics, Equilibrium, 
Wave Mechanics^ Bonding, Etc.); Radioactive Decay Schemes and 
Shielding 
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Cbnununlcatlon : 

Textual Analysis, Text Editing, Speech Synthesis and Analysis, 
Bibliographic Files, Text Editing, CAT 

Earth Science: 

Map making, CAI 

Education: 

CAI Research, Textual Analysis, Statistical Analysis, CAI 
Engineering: 

Simulation, Circuit Analysis, Calculations, Numerical Analysis 
English: 

Textual and Syntax Analysis, Bibliographic Research 

History and Political Science: 
Bibliographic Research 

Home Economics: 

Nutritional Research and Analysis, CAI 

Library Science: 

COM, Computerized Cataloging, Bibliographic Research 

Mathematics: 

Numerical Analysis, Statistical Calculations, (regression 
Analysis, Analysis of Variance, Factor Analysis, Etc.); 
Support of Information. Science Courses, Combinational 
Analysis Studies; CAI 

Modern Languages: 

Syntax Analysis, CAI 

Music: 

Electronic Music 

Nursing: 
CAI 

Physical Education: 
Statistics 

Physics: 

Data Reduction and Analysis, Expansion In Orthogonal Functions, 
Numberlcal Analysis, Statistical- Analysis, Matrix Manipulation 
and Eigenvalve Problems, Signal Averaging, Simulation, Plotting, 
Network Analysis, Plotting and Graphics, Circuit Analysis, CAI 
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Religion and Biblical Languages: 

Textual and Syntax Analysis, Bibliographic Research, CAI 

Secretarial Science: 

Familiarization with Data Processing and Computerized Files, 

CAI, Testing 

Technology and Industrial Education: 

Computer Graphics, Keypunch Training, Computer Operator Train- 
ing, Computer Programmer Training 



h 



A-3 



Appendix B 
NEEDS VERSUS DESIRES 



In order to establish specifications and criteria for computer 
selection. It was necessary £b determine required and useful 
features. The determination was based on extensive meetings with 
faculty members (most departments were consulted) as to needed and 
desired applications. 

Those features found to be required are listed; 

• ANSI COBOL and administrative applications packages. 

• Versatile timesharing (BASIC or XBASIC; ANSI FORTRAN; 
ANSI COBOL). 

9 Load-and-go FORTRAN. 

• Concurrent batch and timesharing capability. 

e Large batch pirocessing capability (at least 100 
K bytes) for at least COBOL and FORTRAN. 

• Batch multiprogramming and spooling capnibillty . 

• Two or more 9 track tape drives. 

• Capable statistical and scientific applications 
packages. 

• Data base management system. 

• Text processing system. 

• Discrete and continuous simulation packages. 

• String manipulation language. 

• CAI language and applications packages. 

• Optical page reader. 

• Digital plotter. 

• CPU to CPU communications capability (not necessarily 
implemented with initial configuration). 

Those features found to be very valuable, although not absolutely 
required, are listed In order of priority; 

• APL. 

• Graphics capability. 

• Digitizer. 

• PERT and Linear Programming application packages. 

• FORMAC. 
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Appendix C 
COST EFFECTIVENESS STUDY FOR AUTOMATING 
THE MUSIC DEPARTMEOT LIBRARY 



Introduction 

This study will briefly address three subjects: a conservative 
statement of costs of present methods of acquiring and c;utaloging 
records, the cost of an automated cataloging system for records and 
scores, and an indication of improvements in services possible through 
automation. 



Costs of Present Methods 

The following listing describes record cataloging operations, and 
corresponding costs, according to present methods: 

In order to prevent duplicate orders, a file of out- 
standing orders must be maintained. Presently, selec- 
tions on only one side of the ordered record are main- 
tained; therefore, if another teacher orders for the flip 
side, a duplicate order may be made. The computer would 
maintain a current listing of all current holdings by 
company and order number as well as a corresponding file 
for records on order with a complete listing of all sel- 
ections on the record. This order file is now kept in 
order by a worker (not librarian) — 3 hours per week, at 
a wage of approximately $1.80, for 40 weeks costs 

$256 per year. . $256.00 

Keeping these records cannot be done in the ac- 
quisition department because the titles on the record 
slip-case are not the ones we use on our cards. We use 
a uniform title, to keep all works score and record to- 
gether for one composer. Only a trained cataloger can 
maintain these records. 

A "New Book List" can be generated easily by the com- 
puter. To provide this, service presently requires man- 
ual effort of a librarian — 1-1/2 hours per month, at a 
wage of approximately $3.00, for 11 months costs $49.50 
per year. 



$49.50 
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After a teacher makes a request, it is easy, with an 
automated system, to check immediately in the current cat 
alogue to see whether or not the record is in the library 
This eliminates the labor of typing an order which Mrs. 
Hammill must then check in the Main Card Catalog — 3 hours 
per week, at a wage of approximately $1.80, for 50 weeks 
costs $270 per year. 

In order to maintain the catalog, one worker pre- 
sently does nothing but type cards — 15 hours per week, 
at a wage of $1.80, costs $1,350 per year. 

A trained librarian must then ch^ck all newly filed 
cards to insure Main Catalog integrity — 4 hours per 
week, at a wage of $3.00, for 50 weeks costs $600 
per year. 

A dated temporary shelf-list card is made for each 
record. Every month this file must be manually checked 
to check that final copies of the catalog cards are made. 
Losses inevitably occur; however, with an automated 
system the card need not be retyped but can be automa- 
tically typed by the computer. The manual process must 
be done by a trained librarian — 2 hours per month, at a 
wage of $3.00, for 12 months costs $72.00 per year. 

Teachers require lists of selected records and cor- 
responding scores for assignment of required listening. 
With an automated system these lists can be prepared 
easily by use of computer sorting techniques. The 
teacher can also see records and corresponding scores 
on the same listing (a service we are not able to pro- 
vide at present) as an aid in making assignments and 
placing material on reserve. Even the present limited 
manual method is laborious and requires the services 
of a trained librarian — 4 hours per week, at a wage of 
$3*00, for 48 weeks costs $576 per year. 

The net cost of operating the manual record 
cataloging system per year is $3,124.00. Note 
that this does not include the cost of cataloging 
scores. 

The cost to acquire and catalog a book is approx- 
imately $5.32. A record in even a single category will 
cost slightly more because a book has a need for only 
four cards — Author, Title, Subject, and Shelf list — 
while a record requires at least five or six cards — 
Composer, Title Subject (1 or 2), Performer (usually 
more than one), and Shelf-list card. Because of the 



need £or more cards, the cost for a record is at least 
$.20 to $.40 higher. Records having different cate- 
gories on each side require a set of cards for each 
category — the equivalent of multiple books. Records 
cost considerably more than a book does to catalog. 
We estimate that a record which costs $3.00 to purchase 
will cost, very conservatively, $7.00 or more to cat- 
alog in our present system. 



Costs of Automated Methods 

The following summary is a conservative estimate of 
costs for an automated cataloging system with an annual 
catalog and comprehensive quarterly supplemental catalogs. 
The subtotals show costs for a system which catalogs only 
records. The final totals show costs for a complete re- 
cord and score cataloging system. 

As usual, there is a one-time costs for creating the 
system and transferring existing files to the computer: 

Programming record cataloging system $1,710.00 

Keypunching and verifying existing record files: 2^437.50 

Computer time for loading files: 20.00 

Total implementation record cataloging system 

cost $4,167.50 



Programming score cataloging system: $ 812.50 

Keypunching and verifying existing score files: 420.00 

Total system Implementation: $5,400.00 
Annual costs would than be quite small: 

Printing all required books annually: $ 167.50 



Three quarterly file updates; 15,50 
Three quarterly sets of catalog supplements: 93^75 
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Total annual cost for record cataloging system: $ 276.75 

Printing score books quarterly: 20>00 

Total annual system cost $ 296.75 
Note that if changes are required or if library 
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accession rate changes significantly, some changes would occur in 
actual costs. 

It is interesting to note that the break-even point on the automated 
record cataloging system versus the manual system occurs almost exactly 
six quarters after it is first placed into operation. After that time 
a very substantial savings will accumulate* 



Improvements in Services 

Although the following comments indicate immediate expected im- 
provements in services or other advantages of the automated system, 
we do not represent the listing to be exhaustive: 

• The automated system offers an easy method for preventing dup- 
licate orders. A current listing of all holdings by manufac- 
turer and order number may be obtained at any time. 

• Duplicate cards may be prepared without delay and without 
laborious manual typing. 

• Listings of holdings in specified categories may be individually 
prepared for use of teacher without delay and for only a few 
dollars. 

• Many clerical errors may be eliminated by use of keypunching 
followed by verification and exploitation of the inherent 
accuracy of the computing system itself. 

r 

' • The expense and lost time in preparing duplicate cards by the 
Xerox process may be avoided by use of supplemental catalogs. 
Up to six copies of catalogs and supplements may be made at 
no additional charge. 

• The supplemental catalog method requires no filing and allows 
users to easily search for required items by visual methods, 

• The "on order" file can be used as a tool in order follow-up as 
well as a source of data for file maintenance. 

• Physical inventory can be made much easier if lists of holdings 
in shelf-order are prepared before commencing inventory. 

• Mrs, Raunio should be able to reduce a what is now significant 
investment of time in clerical work to nil with a significant 
improvement in |>roductivlty, 

• The incremental cost for acquiring and cataloging new record 
holdings should drojp from about $7,00 to about $,22, Simul- 
taneously, the speed and range of services offered by the Music 
Department Library should be improved dramatically. 



Appendix D 
CRITERIA AND SPECIFICATIONS 



In order to meet the academic and business computing needs of Andrews 
University as Identified by the Study Committee, a satisfactory system 
must meet at least the following criteria: 



• Total monthly cost for system and maintenance must not 
exceed a nominal value of $10,000. 

• The system must support concurrerkt timesharing and batch 
processing* 

• The system must support demonstrable ANSI COBOL, BASIC, 
ANSI FORTRAN, and load-and-go FORTRAN compilers. 

o The system must support an adequate simulation and model- 
ing language, and an adequate string manipulation language ♦ 

• The system must support adequate statistical and scientific 
subroutine packages. 

o The system must support an adequate text editor and data 
base management system* 

• The system must be available for delivery prior to June 
15, 1973. 

• The system must support spooling and batch multiprogramming. 
In a dedicated batch mode, user core area must be at 

least 100 K bytes (the minimum necessary to execute many 
standard statistical packages. 

• The timesharing capability must Include excellent file 
security features. 

• The system must support nine track tapes. 

• The proposed system must be capable of expansion to at 
least 256 K bytes main memory, 200 M bytes disk storage, 
and 40 communications ports. 

• The vendor must be able to provldr^ excellent maintenance 
support . 

• The system must support existing applications. 

• The Internal code must be consistent with current In- 
ternal code standards (le. , 8 bit Internal code). 

• The system must include an adequate swapping device. 

• The vendor must present an acceptable conversion plan. 
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Appendix E 
RATING SUMMARY 



The following data were evaluated by the weighted point rating 
scheme. 

Hardware Capabilities 

Main memory capacity (K bytes)? 
Virtual? 

Cycle tiTTiC (ns)? 

Average full word fixed point add time (us)? 
Number of instructions? 
Microprogranmdng? 

Registers available to programmer? 

Registers avail^ible to applications? 

Number of channels? 

Rate (K byte/sec)? 
Multiplex or selector? 
Data rate on each channel (K byte/sec)? 
Aggregate data rate (K byte/sec)? 

Inter leaving? 

Throughput rating? 
Disk subsystem capacity (M bytes)? 

Number of spindles? 

Transfer rate (K byte/sec)? 

Average access time (ms)? 
Swapping device capacity (M bytes)? 

Number of spindles? 

Transfer rate (K bytes/sec)? 

Average access time (ms)? 
Tape subsystem (number of units)? 

Number of tracks? 

Density (bpi)? 

Transfer rate (K bytes/sec)? 

Start-stop time (ms)? 
Card punch (cpm)? 

Number of output stackers? 
Card reader (cpm)? 

Mark sense? 
Printer (1pm)? 

Chain or drum? 

Number of print positions? 
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Character set adequate? 

Cost to change character set? 

Line spacing (6,8, or variable)? 
Front end processor? 

Programmable? 

Memory capacity? 
Communications (mixed codes possible)? 

Maximum transfer rate (bps)? 

RJE available? 

Real time available? 
Maximum ports? 

Reasonable terminals as configured? 
CPU to CPU possible? 

Handlers for RJE to HASP? 

Console (type)? 

Speed (cps)? 

Growth Potential 

Main memory sizes possible (K bytes)? 
Disk memory size per spindle (M bytes)? 

Spindles per controller? 

Maximum number of controllers? 

Maximum with proposed controller (s)? 

Absolute maximum (M bytes)? 
Swapping device size per spindle (M bytes)? 

Spindles per controller? 

Maximum number of controllers? 

Maximum with proposed controller (s)? 

Absolute maximum (M bytes)? 
Tape speeds available (K bytes/sec)? 

Speeds available on proposed controller (s)? 

Can speeds be Intermixed on single controller? 

Densities available on proposed controller (s)? 

Can densities be switch selected? 

Can densities be mixed on single controller? 

Can densities be changed In field? 
Communications (front end available)? 

Number of lines per buffer? 

Number of buffers per controller? 

Proposed buffers? 

Proposed controllers? 

Proposed ports? 

Opeiatit^R System 

Disk files (maximum size)? 
Variable size? 
Indexed sequential? 
Editing features? 
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Access Independent of language? 

Mixed mode creation and access? 

Protection? 
Tape files (maximum size)? 

Variable size? 

Editing features? 

Access independent of language? 

Mixed mode creation and access? 

Protection? 
Batch processing (maximum job streams)? 

Maximum jobs per job stream? 

User I/O blocking? 

Spooling 

CPU to CPU interface possible? 
RJE? 

Ease of JCL use? 
Dynamic resource allocation? 
Operator dependence? 
Concurrent processing? 
Job accounting? 
Real time? 
Swapping control? 
Priority control? 

Changes during execution? 
First release date? 
Current release date? 

Language Processors 

COBOL level? 

Compiler size (K bytes) 

Minimum practical resident (K bytes)? 

Reentrant? 

Generates reentrant code? 
First release date? 
Current release date? 
Gross-reference list? 
Sort? 

Index sequential? 
Direct access? 

Interactive/conversational/batch? 
RJE? 

Diagnostics? 

Core index? 

More corresponding? 

ASCII 
Mixed mode? 
Trace? 
Checkpoint? 
Bit manipulation? 
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Specification of overflow for index sequential? 
FORTRAN level? 

Compiler size (K bytes)? 

Minimum practical resident (K bytes)? 

Reentrant? 

Generates reentrant code? 

First release date? 

Current release date? 

Maximum hardware precision (bits)? 

Complex arithmetic? 

Mixed mode? 

Logical IF? 

Maximum array dimensions/indices? 
Interactive/conversational/batch? 
RJE? 

Diagnostics? 
Functional subscripts? 
Direct access for disk? 
Blocked records? 
Negative indices for DO loops? 
Read end parameters? 
Load-and-go FORTRAN (available)? 
Level? 

Compiler size (K bytes)? 

Minimum practical resident (K bytes)? 

Reentrant? 

Generates reentrant code? 

First release date? 

Current release date? 

Maximum hardware precision (bits)? 

Complex arithmetic? 

Mixed mode? 

Logical IF? 

Maximum array dimensions /indices? 
Interactive/ conversational/batch? 
RJE? 

Diagnostics? 

Functional subscripts? 

Direct access for disk? 

Blocked records? 

Negative indices for DO loops? 

Read end parameters? 
BASIC (available)? 

Extended? 

Subroutines? 

Batch available? 
. Mixed mode? 
ALGOL (available)? 
PL/1 (available)? 
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APL (available)? 
Extended? 

Variable workspace size? 

Mixed mode? 
Discrete simulation language? 
Continuous simulation language? 
String manipulation language? 
Formula manipulation language? 
CAI language? 

Data base management language? 

Conversion» Maintenance and Software Support 

Conversion support (overall evaluation)? 
Vendor man-hours? 
Charge? 
Location? 

Contractural protections? 
Vendor supplied CPU time? 
Charge? 

Location of nearest service center? 
Maintenance support (overall evaluation)? 

Proximity of nearest repairman (miles)? 
Proximity of nearest spare parts storage (miles)? 
Proximity of back-up repairman (miles)? 
Proximity of back-up spare parts depot (miles)? 
Extra shift charges? 
P/M policy? 

Contractural protections? 
Software support (overall evaluation)? 

Range of packages (overall evaluation)? 

Adequacy of packages (overall evaluation)? 

Proximity of nearest systems analyst support (miles)? 

Charges? 
Contractural protections? 

Miscellaneous 

Response to study (overall evaluation)? 
Fonaal training (adequacy)? 
Charges? 

Recommended man-days? 
Continued training support (adequacy)? 
Charges? 
Source? 

Preinstallation support (adequacy)? . 
Site survey? 

Power requirements (voltage, phase, KM and KVA)? 
Air conditioning requirements (tons — include people)? 
Raised flooring or other site modifications? 
Disk pack description? 
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Shipping date? 
Delivery date? 
Expected acceptance date? 

Contractural protections? 

Provision for parallel run on-site? 
Will vendor assume charge? 
Nearest back-up site? 
Upgrade policy? 
Stability of corporation? 

Stability of product line within vendor *s total line? 
Expectations of future product development? 
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Appendix F 
REQUEST FOR BENCHMARK 



Purpose 

The principle purpose of this benchmark study Is to demonstrate 
that certain representative programs can be executed, on the systems 
proposed for installation at Andrews University, by vendors receiving 
this request. Sample production programs, and appropriate test data, 
are also included in order to measure ease of conversion. The secondary 
purpose of the study is to measure compile, execution, and response times 
of representative programs, under controlled circumstances, as an aid 
in assessing processing capabilities of the various proposed systems. 

Andrews University does not suggest that the programs submitted con- 
stitute a comprehensive or representative mix of processing to be accom- 
plished in the future* Rather, the programs were chosen to exercise 
certain compiler and machine features which are of particular Interest. 

Time Frame 

In order to obtain system delivery at a time acceptable to Andrews 
University, an order must be placed no later than the middle of January 
1973. For that reason we must require that results -of this study be In 
the hands of LeRoy Botten absolutely no later than 1:00 p.m. on January 
8, 1973. Please do not request deviations from this policy, none can be 
granted. If for some reason the study can not be completed, please 
return partial results. Results received after the stated time can not 
be considered. 



Evaluation of Results 

Andrews University reserves the iflght to make the final determination 
of the value and usefulness of any oif all parts of the study. In view 
of the commonly understood difficulties of conducting a fair benchmark 
study J we do not Intend to make a final determination necessarily based 
on the results of the benchmark. Nevertheless, we. do desire results 
which can be used to make meaningful comparisons between proposed 
systems. 

Since a relatively short time has been allowed to complete the study, 
we do not expect that a test system will be configured precisely to the 
bid configuration. It is necessary that a full description of the test 
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configuration be forwarded with test results. Results submitted without 
an adequate configuration summary will not be considered * 



One of the most difficult tasks in evaluating a benchmark study is 
determining Ijow variations between bid and test configurations have biased 
results. Realizing that each vendor is in the best position to understand 
any existing biases, we expect that a full disclosure of such biases will 
accompany test results. The cover letter accompanying test results, or 
partial test results, must contain the following statement: 

"(Vendor) certifies the configuration proposed 
for Andrews University will meet or exceed perfor- 
mance standards specified or implied by the attached 
benchmark test results. Where necessary, due to dif- 
ferences between proposed and test configuration, re- 
sults of tests have been adjusted to represent the pro- 
posed configuration performance. All such adjustments 
have been specifically noted. (Vendor) is willing to 
include in the final contract a commitment to rerun 
the benchmark study on the installed system as a final 
acceptance test such that system acceptance by Andrews 
University will be contingent on performance equal or 
better in all respects that performance specified or 
implied by the enclosed test results," 

Please understand the intent of the above paragraph is to help 
prevent a difficulty common to many benchmarks: test systems that are* 
configured to perform better than proposed systems. Vendor cooperation 
with the intent of this paragraph should help Andrews University to 
fairly interpret the results of this benchmark study. Results sub- 
mitted without the above statement will not be considered. 



Programs 

The test programs (described in enclosures) will be identified in 
Che test procedures as follows: 



BASIC 




Bl 


BATCH REGRESSION 


B2 


CONVERSATIONAL REGRESSION 


B3 


RANDOM 


B4 


MATRIX 


B5 


FILES 


B6 


GRAPH 


B7 


ALPHA 


B8 


B CRUNCH 
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ERIC 



COBOL 



CI FILE 

C2 CREATE UPDATE 

C3 PRINT 

C4 COPY 

C5 SORT 

C6 DEMO (data for C2, CREATE) 

C7 DEMO (data for C2, UPDATE) 

FORTRAN 



Fl F CRUNCH 

F2 DIFFRACTION 

F3 FOURIER 

F4 EQNS 

F5 COMPLEX 



Conversion Test . . 

^. 

Record man-hours and systems resources required to convert 
COBOL programs CI, C2, C3, C4, and C5- Provide listings of converted — 
programs. 

Dedicated Machine Performance Tests 

In order to allow controlled timing tests, the following programs are 
to be separately run on a totally dedicated system in the timesharing 
mode: B8, Cl> Fl. Program logic includes print statements which are 
to be used in timing. Using a stopwatch, time the ir.**erval between the 
"RUN" command and start of the output, ''START;" record this interval 
as "compile time." Using a stopwatch, time the interval between the 
start of output, "START," and the completion of output, "STOP;" record 
this interval as "execution time." 

Similarly, the following programs are to be separately run on a 
totally dedicated system in the batch mode: ^B8, CI., Fl. Modify 
program code as necessary to use system interval timer to record "com- 
pile time" and "execution time" as defined in the previous paragraph. 
Record interval timer precision. 

Language Tests 

Make coding changes required to execute Bl, B2, B3, B4, B5, B6, and 
B7 in the timesharing mode. Submit a listing of each program as executed 
and corresponding output. . - 
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Make coding changes required to execute C2, C3, C4, and C5. Submit 
a listing of each program as executed, complle-through-load time for 
each program, and output from C3. Timing to be based on system interval 
timer (make coding changes required for its use with each program) . 

Make coding changes required to execute F2, F3, F4, and F5. Submit 
a listing of each program as executed, compile-through-load time for 
each program, and output from each program. Timing to be based on 
system interval timer (make coding changes required for its use with 
each program) . 



Multiprogramming and Concurrent Processing Tests 

For the background processing in this test two job streams are to be 
used: a precompiled COBOL job stream composed of data C6 and C7 processed 
by precompiled programs C2, C3, and C4; and a sequence of compile-and-go 
runs for F2, F3, F4, and F5. Each of these job streams is to be run in 
the order shown. On completion of each sequence the cycle is to be 
immediately restarted (eg., F2, F3, FA, F5, F2, F3, F4, F5, F2,...)- 
Using the system interval timer, measure the time from start to end of 
each sequence (as requested below) and record as "COBOL sequence time 
(CST)" ami "FORTRAN sequence time (FST)" respectively. Submit a listing 
of each sequence as executed. 



No Timesharing Users 

Record average values of CST and FST with no timesharing users. 

\ 

Five Timesharing Users ' 

Record average values of CST and FST with five timesharing users ^ 
occupied as follows (each user repeats his "assigned" process during 
duration of test phase):. 



\ 
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1 creating and debugging Bl 

2 running B2 ' 

1 creating and debugging CI 

. 1 creating and debugging F5 



Ten Timesharing Users 

Same as for five timesharing users, except two sets of timesharing 
users as described above. 



Fifteen Timesharing Users 

Same) as for five timesharing users , except three sets of five ^ 
timesharing users as described above. 
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Minimum Standards 



Although all parts of the benchmark test are of interest, the time 
allowed for completion is somewhat less than would normally be expected. 
Andrews University is most interested in complete results for the 
Conversion Test, the Language Tests, and at least some demonstration 
of multiprogramming and concurrent (timesharing with batch) operation. 
When these results are assured the other tests should be run. 




\ 



\ 
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Appendix G 
NON-STANDARD CONTRACTURAL ARRANGEMENTS 



Every attempt should be made In contract negotiations to obtain the 
following non-standard protections: 

• Stationing of a customer engineer and storage of essen- 
tial spares at Andrews University (even If we must 
provide office and storage space). 

• Specification of new style memory. 

Q Guarantee of upgrade and trade-in privilege for components 
under Installment purchase plan for at least the first 
eighteen months after Installation* 

• Guarantee of provision for maintenance of all system 
components for the entire life of the Installment 
purchase contract with escalation protection. 

• Non-appropriation clause* 

• Provision for system and software acceptance tests* 

• Conversion and installation non-performance penalties. 

• Guarantee of adequate system analyst support for entire 
life of the installment purchase contract* 

• Guarantee of adequate documentation at fixed cost. 

• Guarantee of adequate training assistance at fixed cost* 

• Right to pay off balance without penalty* 

• Right to transfer equipment to any affiliate of General 
Conference of Seventh-day Adventists without jeopard- 
izing maintenance or systems analyst support* 

• Right to use independent memory or peripheral equipment* 

• Bundling guarantee for entire life of the installment 
purchase contract* 
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Appendix H 
METHODS OF FINANCING 



H.l OUTRIGHT PURCHASE 

Although this method appears to be the least expensive, the cost of 
capital may be Ignored only If the Institution should have a large 
available cash surplus. Purchasing with borrowed money and repaying 
with Inflated dollars may have substantial benefits; however, one must 
consider the reduction In available credit. In any case, It does not 
appear feasible for most universities to make the capital Investment 
necessary to obtain a satisfactory computing system. 

H.2 THIRD PARTY LEASEBACK 

This method of financing consists of purchasing the equipment (in order 
to take advantage of all educational discounts), reselling the equipment 
to a third party (who supplies the capital), and leasing the equipment 
from the third party. The third party anticipates profits from two 
sources: from tax relief not available to a non-profit educational 
institution; and from Interest payments, and possibly, retention of 
residual value. 

11.2.1 Advantages of Leaseback Over Outright Purchase 

The major purpose of leasing is to obtain the use of capital equipment 
without having to make capital expenditures. Lease payments can pro- 
vide a cash flow superior to that of purchase over the early years of 
the equipment's life. Leasing is an effective hedge against inflation; 
however, to be fair, depreciation deductions can suffer a negative 
effect from inflation. 

H.2.2 Advantages of Leaseback Over Traditional Financing 

Leasing may actually give a cheaper rate. This is particularly true 
when lessee cannot take advantage of tax benefits such as depreciation 
and Investment tax credit—the lessor can purchase the equipment, claim 
the tax benefits, and pass' the savings on. Leasing spares the use of 
existing lines of credit and allows full use of borrowing capacity. 
Most leases provide 100% financing — not even a deposit or down payment 
dips into capital. Often even acquisition costs (delivery cost, etc.) 
can be spread over the lease payments. 
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H.3 RENTAL 



The vendor rental agreement offers many of the advantages of third-party 
lease methods; although invariably, at higher costs. The major advan- 
tages of vendor rental agreements are the capability to easily arrange 
for upgrade without having to dispose of existing equipment, and the 
capability to easily arrange for replacement of particular pieces of 
equipment which may be only marginally serviceable. The last advantage 
may be particularly important in case of mechanical equipment subject to 
rapid wear (eg., card punches, printer, or card readers). 

H.4 VENDOR INSTALLMENT PURCHASE 

Most vendors offer an installment purchase plan. Provisions vary 
considerably from vendor to vendor. 
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